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ABSTRACT 



The purpose of this report is to synthesize two other 
evaluation reports ascertaining the effectiveness of the Scope, Sequence & 
Coordination (SS&C) project on ninth and tenth grade student achievement of 
the National Science Education Standards. The overall evaluation utilized a 
time lag, post- test-only design in which the perceptions and performances of 
students studying science using current approaches were compared with those 
of students studying SS&C. The four groups of students were compared on 
several measures administered each year. This report is divided into two 
sections. The first section contrasts results of the Grade 9 and Grade 10 
comparisons of current and SS&C approaches while the second section examines 
results for students who participated in SS&C in both grades. All data 
indicates that both SS&C classes were more inquiry-oriented and more hands-on 
than the comparison classes. The SS&C project had a substantial effect on the 
learning environments in participating science classes, creating environments 
more consistent with the National Science Education Standards. There is 
little evidence that these changes in classroom behavior produced changes in 
motivation or achievement levels in students. Data for students who took SS&C 
for two years support the notion that longer exposure might produce more 
changes in achievement. (Author/PVD) 
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I. EXECUTIVE SUMMARY 



The purpose of this report is to synthesize the other two evaluation reports ascertaining 
the effectiveness of the SS&C project on ninth and tenth grade student achievement of the 
National Science Education Standards (NRC, 1996). The overall evaluation utilized a 
time lag, post-test only design in which the perceptions and performance of students 
studying science using current approaches (the grade 9 and grade 10 students in the 1994- 
95 and 95-96 school years) were compared with those of the students studying SS&C (the 
grade 9 and grade 10 students in the 1995-96 and 96-97 school years). The four groups 
of students were compared on several measures administered each year. (See the grade 9 
and grade 10 evaluation reports and appendices.) This report is divided into two sections: 
the first contrasts the results of the grade 9 and grade 10 comparisons of current and 
SS&C approaches; the second examines the results for students who participated in 
SS&C in both grades 9 and 10. 

There were very few differences between the results found for the grade 9 and grade 10 
students. The psycho-social learning environments of the comparison and SS&C classes 
in both grades 9 and 10 were described using self reported student perceptions from 
interviews and surveys, teacher interviews and surveys, principal interviews, and 
independent classroom observations. All of these data showed very strong and consistent 
evidence that both the grade 9 and grade 10 SS&C classes were more inquiry-oriented 
and more hands-on than the comparison classes. For both grade 9 and grade 10 there 
were two significant differences on the six scale Learning Environment Inventory: 1) 
SS&C classes included more open-ended experiments than comparison classes, and 2) 
SS&C classes were more likely to use an open-ended sequence where they did hands-on 
activities before lectures. The only difference between grades 9 and 10 was that the grade 
9 comparison students reported their science classes as more difficult than the grade 9 
SS&C students while there were no differences in perceived difficulty for the grade 10 
students. There were numerous significant differences favoring SS&C on the type of 
activities used in class for both grades 9 and 10. For example, both grade 9 and grade 10 
students in SS&C reported that they did more experiments with other students, shared 
results of experiments, and interpreted data more often than the comparison students. 
Finally the teacher interviews and the independent observations of the classrooms 
corroborate the student data with both the grade 9 and 10 SS&C classes being more 
student centered than the grade 9 and 10 comparison classes. The differences indicated 
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that the SS&C project helped create a learning environment that was aligned with the 
type of classroom learning environment recommended by the NRC standards. 

Student attitude and motivation toward science was measured using student self report 
data on surveys. Students in grade 9 and 10 SS&C classes were both similar to students 
in grade 9 and 10 comparison classes in their attitudes and beliefs about their science 
classes and in their tendencies to engage in science-related activities outside of class. 
Although the grade 9 SS&C students reported that their classes were more motivating and 
had more “awesome” science experiences than the grade 9 comparison students, this 
difference did not show up for the grade 10 students. 

Student achievement of the NRC content standards was assessed with both paper and 
pencil and hands-on performance tests. The paper and pencil test included both open- 
ended and multiple choice items, while performance tests included five lab stations and a 
full investigation. The students in the grade 9 and 10 SS&C and comparison classes 
performed equally well on all but one subscale of the measures of science achievement. 
The grade 10 comparison group students performed better on the science and technology 
subscale of the multiple choice test than the grade 10 SS&C students. 

The second section of the report presents comparisons between the grade 10 comparison 
group students and the grade 10 SS&C students who had also taken SS&C in grade 9. 
These comparisons were included because SS&C was originally designed to be a four 
year program and therefore there is interest in the effects of prolonged contact. These 
data show that the results for students who participated in SS&C for two years were very 
similar to the results for all grade 10 SS&C students, with a few more differences 
favoring two year SS&C students. 

In terms of the learning environment the students who took SS&C for two years in a row 
reported perceptual differences on all 16 of the items about classroom activities while all 
10th grade SS&C students had perceptions differing from the comparison group students 
on 15 of the 16 items. The difference in motivation was that the students who had SS&C 
for two years reported being more likely to take science in grades 1 1 and 12 than the 
comparison group students, while all 10th grade SS&C students reported being more 
likely to take science only in grade 1 1. The comparisons between the achievement results 
for the students who took SS&C for two years and the grade 10 comparison students 
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showed a difference only on the physical science subscale of the multiple choice test. 
There were no differences on the other scales or measures. The difference on the physical 
science subscale favored the SS&C students. The comparison of all of the grade 10 
SS&C students to the comparison students had shown a difference on the technology 
subscale favoring the comparison students but this difference was no longer significant 
when the students who took SS&C for two years were used as the basis for the 
comparison. 

The data from the students who took SS&C for two years and the grade 10 comparison 
group students were also analyzed by school and sex. The analysis of the learning 
environment by school indicated somewhat mixed findings. As was true in the all grade 
10 student comparisons seven out of the eleven schools reported significant differences 
on at least one learning environment subscale favoring SS&C students. On the other 
hand, only two out of the eleven schools had significant differences on one or more 
subscales favoring comparison students. In the comparisons with all the grade 10 SS&C 
students, four schools had shown differences favoring the comparison students rather than 
two. 

The findings for achievement by school showed the overall finding of no difference was 
apparently due to different patterns of achievement across schools rather than a consistent 
pattern of no differences. Considering all the achievement measures across the eleven 
participating schools, eight differences between the students in the SS&C and comparison 
classes were found. The SS&C students at five schools had scores on one of the 
achievement measures that were significantly higher than the students in the comparison 
classes (six schools for all grade 10 SS&C students), while no schools showed 
differences favoring the comparison students (two schools had shown differences 
favoring the comparison students for all grade 10 SS&C students). There were no 
differences in the comparisons of the perceptions of the learning environment for boys 
and girls for students who had taken SS&C for two years as compared to those who had 
taken it only one year. However, girls who had taken SS&C for two years scored higher 
than comparison group girls on the multiple choice test, while there were no differences 
for girls who had taken SS&C for only one year. 

In summary, the SS&C project had a substantial effect on the learning environments in 
participating science classes, creating environments more consistent with the NRC 
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standards for students in both grades 9 and 10. Unfortunately there is little evidence that 
these substantial changes in classroom behavior produced changes in the motivation or 
achievement levels of the students. There are several reasons why this lack of change 
may have occurred, one of which is that SS&C may need to be implemented over a 
longer time frame to accomplish the intended results in student achievement. The data 
for the students who took SS&C for two years supports the notion that longer exposure 
might produce more changes in achievement. There are also indications of differential 
effects of SS&C by school. These findings suggest that SS&C is certainly no worse than 
traditional science teaching and that it may be better for some students or schools. 
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II. INTRODUCTION 



This report includes the results of supplementary analyses conducted as part of the SS&C 
evaluation. There are several other reports describing the results of this comprehensive 
evaluation effort. Please see those for more detailed information. All analyses in this 
report compare SS&C students to comparison group students. Results are reported on the 
classroom learning environment, student motivational measures and student achievement 
measures, as well as by school and by sex of the students. This report is divided into two 
sections: 1) comparisons of the results for both 9th and 10th grade SS&C, and 2) results 
for students who took two years of SS&C. 

The evaluation used a post-test only, quasi-experimental design. In this approach the 
performance of students studying science using current approaches (the grade 9 and grade 
10 students in the 1994-95 and 95-96 school years) were compared with those of the 
students studying SS&C (the grade 9 and grade 10 students in the 1995-96 and 96-97 
school years). In essence, it was a time-lag design where the prior year's science students 
were compared to the present year's science students. 

Eleven high schools were included in the evaluation. They were selected to be 
representative of the diversity in the US in terms of geographic areas, population and 
race/ethnicity. Three schools were located in Houston, TX; one in Sacramento, CA; one 
in Riverside, CA; two in a suburban/rural area of IA; one in northeast NC; one in 
Kalispell, MT; one in White Plains, NY; and one in Washington, DC. Originally, there 
were thirteen schools in the evaluation, however, one school did not participate in tenth 
grade SS&C and another school dropped out due to lack of student enrollment. There 
were approximately 50 ninth grade and 25 tenth grade science teachers at these schools 
and over 2,000 ninth and 1 ,000 tenth grade students being taught by these teachers each 
year. 

A nested data collection design was used to gather information at each of the eleven 
school sites using ten instruments developed by the evaluation team. Contextual 
variables were assessed using a classroom observation schedule, principal, teacher and 
student interview protocols, a teacher questionnaire, a student questionnaire, and a course 
content survey. Because the major measure of the effectiveness of the SS&C project was 
student achievement, it was examined from several different perspectives. Students 
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answered multiple choice and open ended science content items, participated in a five 
station hands-on laboratory skills test and designed and conducted an experiment. 
Standardized administration and scoring protocols were developed to ensure consistency. 
Descriptions of all instruments, the instrument development process, and the 
psychometric properties of the instruments are included in Appendix B of the 9th grade 
SS&C evaluation report. 

At each school some data were collected from all participating ninth and tenth grade 
students and teachers. In addition three ninth and tenth grade classes at each school were 
targeted for more comprehensive data collection. Site visits were conducted in the spring 
and fall all years except the 1996-97 year (there were not enough funds available for fall 
visits that year). During the fall visits targeted classes were observed and teachers and 
students were interviewed. During the spring visits, the school principals and most of the 
ninth and tenth grade teachers were interviewed and the three target classes were 
observed. In addition most of the ninth and tenth grade students were given a 
questionnaire to complete. Those not taking the questionnaire were 6-12 of the students 
in each target class who were randomly selected to take the science achievement 
performance tests administered by the evaluators. One student from each target class was 
also interviewed. Toward the end of the school year the paper and pencil science 
achievement tests and a course content survey were mailed out to the schools to be 
completed and returned. See Figure 1 for a schematic of the data collection instruments 
and sources of information. 
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▲ Figure 1: Data Collection Instruments and Sources of Information 
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Four questions guided the SS&C evaluation: 

1. Is the learning environment in the SS&C science classes different 
from the learning environment in comparison tenth grade science 
courses? 

2. Are SS&C students more motivated about science than comparison 
students? 

3. Do SS&C students have a better understanding of science concepts 
than comparison students? 

4. Are there differences between SS&C and comparison science classes 
when data are analyzed by school and sex? 



This report addresses two supplementary evaluation questions. 



1. Are there any differences between the grade 9 and grade 10 student 
comparisons of SS&C and comparison group students? 

2. Are there any differences between the grade 10 comparisons students and 
students who participated in SS&C for two years? 



III. COMPARISON OF 9TH AND 10TH GRADE EVALUATION RESULTS 



Su pplementary Evaluation Question #1: 

Are there any differences between the grade 9 and grade 10 student comparisons of 
SS&C and comparison group students? 

Answer to Supplementary Evaluation Question #1: 

Although there are minor differences in the results, the findings for 9th and 10th 
grade were quite similar. The differences found in the comparisons of 9th grade 
SS&C with the comparison grade 9 students were replicated by those of the 10th 
grade students. 

This section includes a comparison of the 9th and 10th grade results for both the SS&C 
years and the non-SS&C years. Class means were used to compare the results whenever 
there were data available from at least four students in the class and there were at least 
four classes available in each group. If these criteria were not met, student means were 
used instead of class mean. These criteria resulted in using class means to analyze the 
student questionnaire, the science literacy test, and the lab skills test. Student means were 
used to analyze the full investigation test. Statistical significance (p < .05) is indicated 
with one asterisks (*) and p < .01 is indicated with two asterisks (**). The few differences 
between the 9th and 10th grade SS&C students are described in the text accompanying 
each graph. 
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■ Learning Environment Inventory 

The 9th and 10th grade results from the learning environment inventory were quite 
similar. In both 9th and 10th grade, SS&C students indicated their classes used a more 
inquiry -oriented sequence, and engaged in more open-ended lab activities than 
comparison classes. The only difference between the two years was that 9th grade 
comparison classes found their comparison classes more difficult than 9th grade SS&C 
classes, while there was no difference in difficulty for 10th graders. (See Figure 2). 

▲ Figure 2: Learning Environment Inventory Results 

□ 9th Grade Comparison 1995 O 10th Grade Comparison 1996 
|v] 9th Grade SS&C 1996 ■ 10th Grade SS&C 1997 
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■ Time Spent on Class Activities 

The differences in responses to items from the student questionnaire on the amount of 
time spent on various class activities were identical for 9th and 10th grade. There were 
significant differences favoring SS&C on 15 of the 16 items for both years showing that 
SS&C clearly created a more hands-on, inquiry-oriented environment. (See Figure 3 and 
4) 

▲ Figure 3: Time Spent on Class Activities 

□ 9th Grade Comparison 1995 |§j 10th Grade Comparison 1996 

□ 9th Grade SS&C 1996 ■ 10th Grade SS&C 1997 
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▲ Figure 4: Time Spent on Class Activities (continued) 
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[1 9th Grade SS&C 1996 
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■ Observation Results 

The comparisons of the observation results for 9th and 10th grade were essentially 
identical. The SS&C classes were seen as more student-centered and less teacher- 
centered than the comparison classes. In other words, the observations of science classes 
corroborated the student self reported data that SS&C classes engaged students in more 
inquiry-oriented, hands-on activities and spent less time using traditional 
lecture/discussion based instruction. (See Figure 5). 

A Figure 5: Observed Time Spent on Activities 

| 9th Grade Comparison 1995 □ 9th Grade SS&C 1996 

9 10th Grade Comparison 1996 | 10th Grade SS&C 1997 
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■ Inclination to Study Science and Science Motivation 

The student questionnaire showed differences for grade 9 and 10 students’ inclination to 
study science and motivation toward science. In 9th grade, more SS&C students than 
comparison students indicated they had an awesome scientific experience and found their 
class motivating. There were no differences between the 9th grade groups on the items 
related to studying more science. At the 10th grade, however, there were no differences 
on the motivation items, but more 10th grade SS&C than comparison students reported 
they intend to take science in 1 1th grade. This difference appears to be mostly due to a 
change in course taking requirements at one of the schools rather than an overall increase 
in individual motivation to study science. 

▲ Figure 6: Inclination to Study Science and Science Motivation 

□ 9th Grade Comparison 1995 || 10th Grade Comparison 1996 

□ 9th Grade Comparison 1995 | 10th Grade SS&C 1997 
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■ Science Literacy Test Results 

The 9th and 10th grade results on the multiple choice portion of the Science Literacy Test 
were identical (neither showed differences) except for the technology sub-scale. Tenth 
grade comparison students scored significantly higher than SS&C students on the 
technology sub-scale, while there was no difference for 9th graders. (See Figure 7) On 
the open-ended portion of the Science Literacy Test there were no differences between 
SS&C and comparison classes at either the 9th or 10th grade level. 

▲ Figure 7: Multiple Choice Results by NRC Standards and Total Score 

□ 9th Grade Comparison 1996 [I] 10th Grade Comparison 1996 

□ 9th Grade SS&C 1997 ■ 10th Grade SS&C 1997 
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■ Hands-on Test Results 

There were no differences in 9th or 10th grade between SS&C and comparison students 
on the hands-on lab skills test. (See Figure 8) There were also no significant differences 
between groups for 9th and 10th graders on the hands-on and written full investigation 
test. 

▲ Figure 8: Hands-On Lab Skills Test by Station and Total Score 

B 9th Grade Comparison 1 995 [“] 9th Grade SS&C 1996 

gg 10th Grade Comparison 1996 | 10th Grade SS&C 1997 
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IV. RESULTS FOR STUDENTS WHO TOOK TWO YEARS OF SS&C 



Supplementary Evaluation Question #2 

Are there any differences between the grade 10 comparison students and students 
who participated in SS&C for two years? 

Answer to Supplementary Evaluation Question #2 

The comparisons of the results for students who took two years of SS&C in contrast 
to the comparison group students are quite similar to the comparisons of the results 
for all grade 10 SS&C students. The few additional findings suggest that studying 
SS&C for two years produced more positive results. 



Because SS&C was originally designed to be a four year science program, the effect of 
participating in the program for more than one year is of interest. Therefore this section 
presents the results for students who participated in SS&C in both grades 9 and 10. As in 
the 9th grade evaluation, class means were used to compare the results whenever there 
were data available from at least four students in the class and there were at least four 
classes available in each group. If these criteria were not met, student means were used 
instead of class mean. These criteria resulted in using class means to analyze the student 
questionnaire, the science literacy test, and the lab skills test. Student means were used to 
analyze the full investigation test. All analyses by school also used student means 
because there were less than four classes available at some schools. Statistical 
significance (p < .05) is indicated with one asterisks (*) and p < .01 is indicated with two 
asterisks (**). The few differences between all 10th grade SS&C students and those 
students who took two years of SS&C are described in the text accompanying each graph. 
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■ Learning Environment Results 

The comparisons of the LEI results for students who took two years of SS&C were 
identical to the results for all 10th grade SS&C classes. Specifically, students who had 
two years of SS&C found: 1) the sequence in their classes more inquiry-oriented, and 2) 
the lab activities more open-ended than comparison classes (See Figure 1 1 and Table 1 in 
the Appendix to this report). 

▲ Figure 11: Learning Environment Results 

■ 10th Grade □ 10th Grade 
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■ Time Spent on Class Activities 

There was one difference in the results for 10th grade students who took SS&C for two 
years and for all 10th grade SS&C students. For those students who took SS&C for two 
years, there were differences favoring SS&C on all 16 items about class activities (See 
Figure 12 and Table 2). For all SS&C grade 10 students there were differences on only 
15 of the 16 items (the non significant one was amount of time watching video and film) 
Both sets of SS&C students reported their classes as having more inquiry-oriented 
activities than the students in the comparison classes. 

▲ Figure 12: Time Spent on Class Activities 

| 10th Grade [H] 10th Grade 
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■ Students’ Inclination to Study Science and Science Motivation 

There were differences in the comparisons on items measuring inclination to continue 
studying science but no differences on the items addressing motivation toward science 
(See Figure 13 and Table 3). Students who had two years of SS&C indicated they were 
more inclined to take science in 1 1th and 12th grade than comparison students. These 
results are slightly more positive than the results for all 10th grade SS&C students, who 
only indicated an inclination to take more science in 1 1th grade. Also the inclination to 
take science in grade 12 was not affected by the changes in the science course 
requirements at one of the schools since the requirement change was from two years of 
science to three years of science not to four. 

▲ Figure 13: Inclination to Study Science and Science Motivation 

| 10th Grade Q 10th Grade 

Comparison 1996 SS&C 1997 
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■ Science Literacy Test Results 

On the multiple choice science literacy test there were two changes in the differences 
between SS&C and comparison classes. Two year SS&C students scored significantly 
higher on the physical science subscale than did comparison students and the difference 
found in the comparison with all SS&C grade 10 students for the technology scale 
favoring the comparison students disappeared (See Figure 14 and Table 6). On the open- 
ended items, the results for two year SS&C students were identical to the results for all 
10th grade SS&C students; namely, there were no differences between SS&C and 
comparison classes. 

▲ Figure 14: Multiple Choice Results by NRC Standard 
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■ Hands On Lab Skills Test Results 

As on the results for all 10th grade SS&C students, there were no differences between 
two year SS&C and comparison classes on the hands on lab skills performance test (See 
Figure 15 and Table 8). There were also no changes in the findings of no differences on 
the hands-on or the written full investigation test. (See Table 9 and Table 10). 



Figure 15: Hands-on Lab Skills Test Results by Station 
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■ Results by Site 

Seven of the eleven participating schools showed significant differences favoring the 
SS&C classes on at least one learning environment inventory subscale, two of the eleven 
schools had no differences, and two of the eleven schools had differences favoring the 
comparison classes (See Table 11). In the comparisons with all the grade 10 SS&C 
students although seven of the eleven schools had shown differences favoring SS&C, 
four schools had shown differences favoring the comparison students rather than two. 
These results suggest that the SS&C project altered the learning environment at the 
majority of schools and that the two year time frame may have been more successful in 
effecting these alterations. 

The findings for achievement by school showed the overall finding of no difference was 
apparently due to different patterns of achievement across schools rather than a consistent 
pattern of no differences. Considering all the achievement measures across the eleven 
participating schools, five differences between the students in the two year SS&C and 
comparison classes were found. The two year SS&C students at five schools had scores 
on one of the achievement measures that were significantly higher than the students in the 
comparison classes (six schools for the all grade 10 SS&C student comparisons), while 
no schools showed differences favoring the comparison students (two schools had shown 
differences favoring the comparison students with all grade 10 SS&C students.) There 
were significant differences favoring two year SS&C at two schools on the multiple 
choice questions, at one school on the open ended questions, and at two schools on the 
hands-on lab skills test; all other schools showed no difference between two year SS&C 
and comparison classes (See Tables 12-14). 

■ Results by Sex of Student 

There were differences between two year SS&C and comparison classes on only one 
subscale on the learning environment inventory when analyzed by sex (See Table 15). 
Both male and female SS&C students found more inquiry -oriented sequencing in their 
classes than did comparison students. This same difference was found when the data 
were analyzed for all 10th grade SS&C students. 

Although the comparison of all SS&C students and comparison group students had 
shown no differences for sex of the students on any achievement measures, the 
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comparisons with the two year SS&C students showed that the female SS&C students 
scored significantly higher than female comparison students on the multiple choice 
portion of the science literacy test (See Table 16). There were no significant differences 
by sex between the SS&C and comparison classes on the other achievement measures 
(See Tables 17 - 20). 



V. CONCLUDING REMARKS 



The SS&C project was funded by the National Science Foundation as a teacher 
enhancement effort supported by curricular materials. The guiding principles of the 
curriculum are providing every science subject every year, explicitly taking into account 
students' prior knowledge, providing a sequence of content from concrete experiences and 
descriptive expression to abstract symbolism and quantitative expression, revisiting 
concepts, principles, and theories at successively higher levels of abstraction, and 
coordinating learning in the four science subjects so as to interrelate basic concepts and 
principles. The project was committed to developing a curriculum that produces students 
that are better prepared according to the National Research Council (NRC) science 
education standards. The NRC content standards served as the guidelines for the 
development of the assessment instruments used in the evaluation. This means that the 
SS&C project was evaluated on how well the curriculum prepared students based on the 
standards not on the specific goals or objectives of SS&C. 

This summatively oriented evaluation effort was unique in that it gathered a variety of 
information about student outcomes as well as about classroom behaviors. The 
classroom information was obtained from different sources, i.e., students, teachers and 
observers and the student outcome data was obtained from a variety of assessment 
formats, i.e., multiple choice, open-ended, laboratory stations and full investigations. The 
evaluation results provided some interesting insights into teacher enhancement efforts and 
raised several important questions. 

Learning Environments 

The data show that the classrooms were significantly changed by the enhancement effort. 
The learning environment in the SS&C classes was significantly more in line with the 
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types of classes recommended in the NRC standards in terms of more student interaction, 
more hands on activities, more inquiry, and less lecture. These differences were 
corroborated by the observations, by teacher self report and by student opinionnaires. 

The SS&C teachers produced class environments that explicitly took into account 
students' prior knowledge and provided a sequence of content from concrete experiences 
to conceptualization just as was expected from the curriculum. The classes also covered 
all four areas of science, life, earth, chemistry and physics with some degree of 
coordination among them. 



The following two scenarios capture some of the differences between comparison and 
SS&C science classes. 

Comparison Scenario: 

Mr. Watkins had a good earth science lesson planned for the day and he knew the 
students would enjoy it. The students would be working on an experiment and he had 
all his materials placed conveniently on the lab tables. Mr. Watkins stood in the 
doorway and welcomed the incoming students and when the bell rang he took 
attendance. He began the class with a rhetorical question framing the day's activities 
which he then answered using overheads and models. He asked several students 
questions recalling previous earth science material that supplemented his presentation. 
After presenting the concept, he passed out the lab sheets and had one of the students 
read it to the rest of the class. He commented on any difficult parts and demonstrated 
procedures as the students followed along. When all questions were answered the 
students moved to the lab area and conducted their experiment sharing equipment in 
pairs. As the students completed the lab work, they went back to their seats and 
answered the questions on the lab sheet. Mr. Watkins then went over the questions 
with the whole group calling on students to read their answers. He reviewed the 
vocabulary and pointed out how the lab results confirmed the concept he had 
presented earlier. He then had the students restate the concept in their own terms and 
ended the class. 

SS&C Scenario: 

Ms. Moore was looking forward to her class. There was an interesting earth science 
question to investigate and she was not sure what the students would find out. She 
had sets of materials out on the back shelf that the students could use. She welcomed 
the students as they came in and had them sit in their lab groups. She stated the topic 
for today's class and asked the students what they knew about it and if anything from 
the life science portion of the class related to this topic. Finally, she asked the 
students a question and asked how they might go about finding the answer. After 
listening to a few suggestions, she directed the students to use the materials to 
determine an answer. The students worked in their groups with Ms. Moore helping 
out as necessary when the group members could not answer their own questions. 
When all the groups had finished collecting their data, Ms. Moore asked them to 
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report to each other what they had found out. The groups shared their results and 
their thoughts on answers to the lab question. There was a disagreement between two 
of the groups which Ms. Moore suggested they could check on by doing some more, 
lab work the next day. The students left the class debating their answers. 

There were costs and benefits to these changes to the classroom environment. The 
student interviews and opinionnaires showed that the students were quite satisfied with 
their classes. The students thought the classes were fun, not too hard and that the best 
things were the activities. The teacher interviews, however, revealed mixed feelings 
about the implementation effort. The issues coalesced into three areas, materials, 
presentation method and content. 

All of the teachers felt somewhat overwhelmed by the SS&C curricular requirements. 

The materials were being developed/revised as the teachers were using them so there was 
not a "tried and true" way to go with all the glitches highlighted and compensated for. 
This lack of organization did, however, tend to promote individual modification and 
adaptation to the sites' needs. In addition to organizing and selecting curricular materials, 
the teachers felt they had to spend unreasonable amounts of time preparing hands-on 
laboratory materials for the students to use. Furthermore teachers often reported having to 
buy materials on their own or having to use materials that were not quite right. 

Teachers thought the SS&C hands-on approach and having the students use the materials 
before the concepts were presented were good ideas, however, they felt it was just too 
difficult to do as often as the curriculum recommended. Many teachers felt that the 
students needed more help to understand the concepts. They felt they needed the 
concepts explained to them and that this sequence of teaching would result in the students 
not really understanding the concepts and not learning the necessary facts and vocabulary. 

Teachers felt that the coordination among the sciences was weak. They felt the students 
got a little bit of earth science and then were off to something else in life science or 
physical science. They also felt the concepts were not covered in enough depth. In grade 
10 situations where one teacher was teaching all of the sciences, some of the teachers felt 
that they did not have enough background knowledge. On the other hand in sites where 
the SS&C classes were taught by several teachers with the appropriate content expertise, 
some of the teachers felt that they did not get to know their students well enough and that 
it was difficult to coordinate among the science disciplines. 
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Motivation 

It would seem that the most immediate effect of more stimulating environments would be 
improved student attitudes toward science. That, however, does not appear to have been 
the case. The students in the first year of SS&C did seem to think their classes were more 
motivating. By the time the students were in the second year of SS&C, they were blase 
about it. This seems somewhat unusual given the more involving classes. Perhaps the 
other things in the students’ lives are so compelling that the minor motivational changes 
possible within the standard school setting were just overwhelmed. Perhaps the positive 
first year results occurred because SS&C was so different from standard science classes. 
The students were slightly more interested, but this novelty effect wore off by the second 
year. This interpretation is supported by the results of the old ISCS materials which were 
completely lab based and which the students really loved for a while. They found, 
however, that after some time the students were asking their teachers to lecture to them 
just for a change of pace. SS&C seems less one sided than ISCS, since it involves group 
work and class discussions, but the questions about novelty and motivation remain the 
same. Why should students be excited about science when it is only peripherally related 
to their real existence and there are so many more interesting things outside of school? 

Achievement 

A causal model of teacher enhancement would posit changes in student achievement 
based on the significant changes in the learning environment, i.e., if the teachers were 
indeed teaching better, the students should perform better. This effect should be 
especially pronounced if the students had experienced the environment for a long period 
of time, e.g., two years in a row. This, however, was not the case. There were no major 
differences in achievement as measured by the variety of assessment formats even when 
the students had taken SS&C for two years in a row except perhaps in the physical 
sciences. It must be kept in mind that the achievement tests were not tied to the specific 
content covered in the SS&C curriculum. Tests were tied to the NRC standards and 
therefore were a very broad look at student achievement not a look that was designed to 
determine if a student learned what was taught. 

It is difficult to explain the lack of an effect given the strong changes in the learning 
environment. It is possible that the tests were invalid. This seems unlikely for several 
reasons. The psychometric qualities of the tests are strong; the scales show strong factor 
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structure and the reliabilities are high. The items were selected from existing and highly 
valid tests so the items themselves should be valid. Independent science education 
experts examined the tests and stated they matched the standards they were designed to 
measure. The tests show expected results in terms of higher scores for students who have 
higher grades in school, standard patterns for achievement for different ethnic subsets and 
gain in achievement for the sample overall from year 9 to year 10. 

Another possible explanation is that the learning environment has less effect on student 
performance than we would like to believe. This notion is supported by the fact that there 
were no differences in achievement even when students had studied a particular science 
area for the entire year. For example, in the grade ten comparison group sample almost all 
of the students took life science for the whole school year, yet the comparison group 
students did not score higher in life science than the SS&C students who only had life 
science part of the time. Also the SS&C students who had studied earth sciences did not 
perform better than the comparison group students who had not studied those areas. 
Perhaps the real key to student outcome is in the student him or herself. Certainly this 
fits with motivational and social context theory. It may be unreasonable to expect a 
teacher enhancement effort to have any broad effect on student outcomes and that efforts 
should only be assessed on whether the teachers changed their behaviors and whether the 
students learned material specific to the enhancement. 

In summary, the SS&C project had a substantial effect on the learning environments in 
participating science classes, creating environments more consistent with the NRC 
standards in both grades 9 and 10. Unfortunately there is little evidence that these 
substantial changes in classroom behavior produced changes in the motivation or 
achievement levels of the students. There are several reasons why this lack of change 
may have occurred, one of which is that SS&C may need to be implemented over a 
longer time frame to accomplish the intended results in student achievement. The data 
for the students who took SS&C for two years supports the notion that longer exposure 
might produce more changes in achievement. There are also differential effects of SS&C 
by school. These findings suggest that SS&C is certainly no worse than traditional 
science teaching and that it may be better for some students and schools. 
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